-
Notifications
You must be signed in to change notification settings - Fork 183
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
perf: cache activation environment variables #1832
Conversation
Thanks @borchero, cool feature. I'm a little worried about the invalidation as on windows this might not work as well. @baszalmstra can you give your feedback. |
I like this a lot! Thanks for the contribution. The biggest problem is that some of these environment variables are initialized from system environment variables. For instance, when you add the compiler activation scripts on Windows the path to a specific MSVC compiler is added. However, when a new update of the compiler is installed (which happens from time to time) this path is no longer valid. The same goes for the system PATH variable, this can be updated but the changes would not be reflected in our cached entry. I think it would be good to add a time-to-live to the cache entry so that the cache entry will be recomputed after a period of time (30 minutes?). Maybe we can also invalidate the cache if certain environment variables no longer match what they were when the cache was created? I think the PATH variable is a prime example. The entries should also be cleared on |
I'm not sure this is the best solution to be honest. I think it introduces hard-to-debug issues that are non-deterministic over time 👀 That being said, I think it's important that we identify a set of items that need to be hashed. The lockfile is an obvious one. Using the |
Yeah I agree a TTL is very much suboptimal. Maybe we can start by identifying a larger set of environment variables to use as input? I guess its less anoying when you have a stale cache than an invalid one. |
That'd be great 😄 do you have any proposal where to start with this? I personally know very little about Windows and that's usually where most issues arise 🙃 |
If you start the implementation and have a list somewhere Ill investigate what we need on windows! |
@borchero are you planning to work on this? |
Most of this code is copied from prefix-dev#1832 build by @borchero but the merge conflicts made me copy it onto main by hand.
Sorry I didn't get back here @ruben-arts, I did not get the opportunity to work on this recently. Thanks for taking over the PR! I will close this one :) |
Motivation
Resolves #973.
This PR obviously misses some tests, I just wanted to check in early if this goes into the right direction.